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Abstract 

While problem-based learning (PBL) is widely regarded as an effective instructional approach, 
research on it has largely been restricted to advanced and/or gifted learners. This study examined the 
potential of a hypermedia based expert tool to scaffold regular education sixth graders engaged in a 
problem-based learning (PBL) program. Participants completed Alien Rescue , a hypermedia PBL 
environment on the solar system. The tool under investigation offered students interactive video of an 
expert modeling his cognitive processes as he performed tasks relevant to the development of a solution 
to the central problem of the PBL unit Two other versions of the expert tool were developed to isolate 
the effect of the cognitive modeling. Results suggest that the cognitive modeling offered through the 
expert tool brought the way students worked during periods of self-directed study into line with expert 
actions and impacted the quality of the rationales students wrote for their solutions. 



Introduction 

The recent shift within the field of instructional technology to design based on a constructivist 
framework has led to a renewed interest in problem-based learning (PBL), with Savery and Duffy going 
so far as to argue that PBL is the instructional approach that best realizes the tenets of constructivism [1]. 
In PBL, learning occurs as a result of students 1 efforts to solve a complex problem. Instruction begins 
with the presentation of a problem situation, and in order to develop a solution, students identify learning 
needs, locate resources to meet those needs, and apply their learning to the problem situation. 

Proponents of PBL have expounded on its many potential benefits, among them improved self-directed 
learning [2, 3], development of problem-solving skills [4], and a positive attitude toward learning through 
this approach [5]. 

While PBL offers many possible benefits, there are several difficulties with this approach that 
may complicate its general use in K-12 settings. Much of the research on PBL has focused on gifted and 
mature learners, making it difficult to extrapolate their findings to a wide range of learners. Hmelo and 
Ferrari even suggest that PBL is more suitable for use with the gifted than with regular education students 
because the former possess more sophisticated learning strategies, and are more reflective and 
intrinsically motivated than their regular education peers [6]. However, the benefits of PBL represent 
valuable educational goals for all learners, not just for the gifted. An examination of the potential 
difficulties average learners may encounter during PBL can lead to the development of supports that 
could make this approach viable in more classrooms. 

Self-directed study poses one such difficulty. Extensive periods of self-directed study are a 
fundamental feature of PBL; students must determine what information they need, what resources to use 
to gather it, how to organize it, and the sequence in which to perform the assorted tasks necessary to solve 
the problem. Learners are typically allowed to pursue blind alleys in their problem-solving efforts 
without interference on the part of the teacher [4]. Even in medical programs, learners initially have 
difficulty with the lack of externally imposed structure and responsibility for self-directed study [3]. 
Young learners of average ability may lack the skills to identify pertinent learning needs and the resources 
that can meet them. They may have less developed planning skills and be less able to reflect upon their 
efforts and change them when necessary. Supporting students’ self-directed study during PBL is 
complicated by the fact that students’ activities are context specific. The activities that occur during self- 
directed study vary from problem to problem, as students gather and interpret information in ways 
particular to a given problem situation. 

Classroom teachers can provide some support, but PBL may place overwhelming demands on the 
teacher’s time. The role of the teacher differs dramatically in a PBL class from that of the traditional 
classroom, where the teacher determines the content and sequence of the objectives to be met, and 
provides direct instruction. During PBL, the role of the classroom teacher is generally regarded as that of 
a facilitator, helping learning to occur, but controlling neither its content nor its process. Depending on 
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the abilities, maturity, and needs of the learners, facilitators typically probe students’ comprehension in 
order to diagnose misconceptions and identify knowledge deficiencies, monitor group processes to insure 
helpful interactions and participation by all learners [7], provide metacognitive coaching to help students 
to examine their thinking and internalize the habit of questioning [3, 8], and make moment to moment 
decisions about how to best facilitate the PBL process in order to meet the idiosyncratic needs of 
individual students and classes [6], As important as what the teacher does is what the teacher does not 
do. PBL is designed to be student-centered; a teacher who is overly directive can deny students the 
opportunity to reason and engage in self-directed study, thus undermining major goals of the PBL 
approach. On the other hand, a teacher who is too passive will fail to guide students to consider all the 
steps in the reasoning process and develop the habit of questioning. Students will flounder, making the 
PBL experience less productive and enjoyable than it should be. Barrows cautions that the skillfulness of 
the teacher has a major impact on the success of PBL [9]. The potential of PBL to promote students’ 
development of problem-solving and self-directed learning skills is dependent on both a student-centered 
approach and sufficient support. This tightrope may be too narrow for teachers inexperienced with PBL. 

The difficulty of this balancing act is exacerbated by the lack of self-directed study skills on the 
part of young, average learners, which may tempt teachers to “take charge,” falling hack on the traditional 
instructional patterns of teacher-centered education. As a result the teacher may unwittingly sabotage the 
opportunity PBL offers to make learning student-centered and help students develop skills of self-directed 
study. In this light, PBL may become a viable approach for more K-12 classrooms if PBL instructional 
materials incorporate support for students’ work during self-directed study. Not only can these materials 
share with the teacher the responsibility for delivering the many types of support learners need, they can 
also be fine tuned to be as non-directive as possible. This can relieve the teacher of one of the more 
subtle and complicated types of support students may need, leaving him or her to focus on needs 
particular to his or her students. 

Cognitive Apprenticeship Support for Problem-Based Learning 

In her critique of PBL, Williams concluded that a weakness of this approach is that, in the 
absence of good models of problem solving, students must acquire skills through trial and error [3], Asa 
result, novices sometimes flounder, adopt ineffective strategies that interfere with later learning, and 
experience frustration. She recommended that learners be given the opportunity to see experts in action, 
using domain knowledge and strategies to solve problems. The type of support she advocates constitutes 
a cognitive apprenticeship, wherein learners work with experts within a domain, gradually becoming 
aware of and internalizing the cognitive processes of those experts [10]. In a cognitive apprenticeship, the 
internal processes that normally remain hidden are externalized. As both experts and novices work 
together on a task, they can observe and compare their cognitions in order to help the novice develop and 
apply the same knowledge and skills as the expert. 

To be effective, experts must provide just the level of support needed by the novice to perform 
the task; like the narrow balance required of the PBL facilitator, either too much or too little support may 
limit the apprentice’s opportunity to participate effectively in the community. Drawing on the work of A. 
L. Brown [11], Derry and Lesgold propose a three-stage model in which the role of the expert mentor 
changes as the learner progresses toward proficient participation [12]. In the first stage, the expert models 
his or her problem-solving, thinking aloud so that the apprentice has the opportunity to see what cognitive 
and metacognitive strategies the expert brings to bear on a task. In stage two, the expert and apprentice 
work together, with the expert modeling, coaching, and prompting only as the apprentice falters or 
requests support. In stage three, the apprentice is given control of the task, and the expert’s support is 
limited to that of questioning and serving as a supportive audience. Learners new to a domain or who 
possess limited problem-solving and self-directed study skills would fall in the first stage of this model. 
This suggests that, for these learners, modeling of the cognitive and metacognitive processes an expert 
brings to bear in addressing the task posed by the PBL environment may be an appropriate support 
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Modeling is a teaching method wherein the learner is given an opportunity to observe the internal 
cognitive processes of an expert as he or she problem-solves [10]. Usually this is done by having the 
expert do “think alouds” as he or she deals with a problem or by sharing stories of the thinking processes 
he or she has used to solve problems in the past. This extemalization of these normally internal cognitive 
events allows students to see how an expert uses domain specific knowledge and a range of problem- 
solving strategies to perform tasks within a given context. This helps students to develop conceptual 
models of the processes that are required to accomplish the task. Through the provision of modeling 
during PBL, learners may work more effectively and experience less frustration. 

Using Hypermedia to Offer Expert Modeling 

Hypermedia programs are characterized by a coordinated use of multiple media and interactivity. 
These features mean that hypermedia programs offer a number of affordances that make them an effective 
medium through which to deliver and support PBL. For example, the non-linear access to information 
afforded by hypermedia means that a program can contain all information needed to solve a problem 
without suggesting a sequence of learning within the PBL unit. The multiple media allows a degree of 
realism and richness in the presentation of a problem scenario. The study reported here describes and 
examines a tool that capitalizes on the affordances of hypermedia to offer cognitive apprenticeship 
support to learners engaged in PBL. 

Through the use of audio, video, animation, and context-sensitive timing, a hypermedia program 
can be designed to offer modeling of pertinent strategies as students are engaged in problem-solving by 
providing expert opinions at appropriate points throughout the program. These “experts” can be 
hypermedia-based characters who pop up at key points within the program to share relevant stories or 
explain useful strategies as in the case of this study. While the classroom teacher could also model his or 
her cognitive processes for the class, presentation of this modeling through hypermedia may have some 
distinct advantages. First, hypermedia delivered modeling of expert processes can be offered in a “just- 
in-time” manner. The modeling can be time or location sensitive; it can be offered after students have 
spent a certain amount of time in the program, or only when they are using certain tools within the 
program. Second, delivery of the modeling can be under program or user control, depending on design 
considerations. In other words, the instructional designer can determine that some modeling is essential, 
and design the program so the students are coerced into viewing it. On the other hand, it is possible to 
leave control over access to the learner, allowing him or her to determine if and when expert modeling is 
helpful. Third, hypermedia is endlessly patient. Students can review the expert’s cognitive processes as 
many times as they like, controlling which parts they replay. Finally, the hypermedia delivered modeling 
can be developed by a team of instructional designers, meaning that it can draw on the expertise of 
several educators rather than a single classroom teacher. The teacher, whose experience with the program 
may be limited, may not yet possess the expertise in addressing the problem situation required for helpful 
modeling. The teacher may lack the skill of making his or her own cognitive processes explicit, focusing 
on what he or she does instead of why those choices are useful. He or she may also be tempted to become 
overly directive, telling students to perform tasks in a particular sequence and only in the manner 
modeled. The hypermedia based support can be carefully designed to avoid authoritative language that 
may limit students’ sense of control over their process. 

Though Collins argued a decade ago that hypermedia could be used to offer cognitive 
apprenticeship style support, thus realizing types of learning environments that have not been possible of 
cost-effective in the past, limited attempts have been made to utilize the affordances of hypermedia in this 
manner [13]. A major line of research on using hypermedia to provide cognitive apprenticeship support 
focuses on intelligent tutoring systems (ITSs). While ITSs come in a variety of flavors, their primary goal 
is typically to provide coaching, a type of cognitive apprenticeship support in which the teacher (in this 
case the computer) watches learners as they are engaged in a given task, then gives hints and feedback 
which are designed to improve their performance. The drawback to ITSs is that they are time-consuming 
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and expensive to develop [14], Alternative ways of exploiting hypermedia to provide cognitive 
apprenticeship support is warranted. 

The Purpose of the Study 

Two concerns were raised in the review of literature. First, despite the evidence of the benefits of 
problem-based learning, its use has largely been restricted to mature and/or gifted learners, offering 
limited challenge to the perception that PBL is not a suitable approach for young students or regular 
education classrooms. Second, the self-directed study component of PBL can be difficult for young 
learners, and the support they need to develop the necessary skills for self-directed study can place an 
overwhelming burden on a classroom teacher. This study seeks to determine if a hypermedia based tool 
can share the responsibility for supporting learners during PBL, making this approach a viable and 
beneficial option for middle school students in regular education classes. This tool offers video-based 
modeling of an expert’s cognitive processes as he engages in the same activities that are useful to students 
participating in a PBL program, and its design was informed by the literature on cognitive apprenticeship. 

This expert tool has two functions. First, it is designed to give learners ideas about useful 
activities to engage in during the periods of self-directed study that form an important part of the PBL 
process. Learners who are inexperienced with self-directed study may lack the skills to put this time to 
good use, and may even flounder, unable to determine how to get started or what to do if they get stuck. 

By modeling the activity of an expert, this tool is designed to help learners use these periods of self- 
directed study effectively. Second, the expert models a variety of skills useful in problem-solving, such 
as reading to gather pertinent information, making connections between pieces of information, recording 
information for later use, forming hypotheses, asking questions, developing a plan to collect data, and 
supporting a solution plan with a rationale. As learners view the video segments provided in the expert 
tool, they can see how an expert would apply these problem-solving skills to the same task that they face. 
This study therefore sought to determine whether or not the expert tool impacted the way learners worked 
during periods of self-directed study, and if so, whether or not this support resulted in better solutions to 
the problem presented in a PBL unit 

Additionally, a potential negative effect of the expert tool on learner attitude was examined. 

While providing modeling of cognitive processes through the expert, a concern is that such an expert tool 
may interfere with students’ sense of ownership over their process, which could negatively impact both 
their attitude toward the environment and their interest in participating in similar ones. Therefore, this 
study examined the impact of the expert modeling tool on learners’ attitude toward the environment 

Design 

Research Questions 

The research questions for this study focus on the effects of expert cognitive modeling on 
learners’ performance within and attitudes toward the learning environment: 

Does hypermedia based expert modeling affect learners’ work during periods of self-directed study in 
PBL? 

Does hypermedia based expert modeling affect the quality of the rationale students develop for their 
solution to the central problem posed by the PBL unit? 

Does hypermedia based expert modeling affect learners’ attitude toward a PBL environment? 

Sample 

Sixty-six students in three intact sixth grade science classes at a suburban middle school in the 
southwestern United States participated in the study. These three classes were taught by the same teacher. 
Classes were randomly assigned to the treatment conditions. 

The ethnic makeup of the classes reflected the school at large. 77% of the participants were 
Caucasian, 1 5% were Hispanic, 5% were African American, and 3% were Asian. The classes were split 
almost evenly by gender, with two more girls than boys participating in the study. 
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Instructional Materials 

The hypermedia program used in this study was Alien Rescue, a problem-based learning 
environment for use in sixth grade science classes. The science fiction premise of this program places 
students in the role of young scientists aboard a newly operational international space station where they 
are part of a worldwide effort to rescue alien life forms. The opening scenario of Alien Rescue informs 
students that an alien ship has entered Earth orbit and sent a distress message explaining that the aliens 
aboard the ship are the survivors of a distant solar system that was destroyed by the explosion of a nearby 
star. An accident during their voyage has left their computer damaged and the aliens are in suspended 
animation because their ship can no longer provide life support. Students are assigned the task of 
recommending worlds in our solar system that would be suitable homes for each of the six species of 
aliens aboard the ship. To accomplish this goal, students learn about the planets and large moons of our 
solar system by searching existing databases and using a simulation within the program that allows them 
to design and launch probes and interpret the data these simulated probes collect While the primary 
learning objectives of Alien Rescue focus on astronomy and space travel, the program offers ties to other 
areas of the curriculum, including life science and mathematics. The program provides several 
opportunities for writing: students record notes in an online notebook, develop mission statements for the 
probes they design, and write rationales for the solutions they develop. 

The focus of this study is on one tool provided within the Alien Rescue environment: the expert 
tool. This tool offers video segments of a character who is an expert scientist engaged in the same task as 
learners, that of finding homes for the aliens. The purpose of this tool is to show learners strategies that 
an expert would use while addressing this problem. It is designed to provide just-in-time support as 
learners are engaged in various tasks so as to make their periods of self-directed study more productive 
and help them to develop better solutions to the problem. 

Treatment Conditions 

Three versions of expert support were used as the treatment variable. In the modeling condition, 
an expert modeled two general tasks: his cognitive processes during problem-solving activities and tool 
functionality. In the didactic condition, the same expert explained tool functionality and offered tips on 
how learners should work during self-directed study, but did not model his application of these strategies 
to the tasks at hand. In the help condition, the expert explained tool functionality outside of the context of 
their use in problem-solving activities but did not offer any advice about how students should work. By 
comparing these three conditions, it is possible to isolate the effect of the cognitive modeling on learners’ 
self-directed study and problem-solving. 

The Modeling Condition 

In the modeling condition, an expert modeled his cognitive processes as he engaged in problem- 
solving activities within the Alien Rescue environment. These activities are ones that occurred as 
students were engaged in self-directed study. This modeling was delivered in video format “just-in-time” 
to students as they were working within the section of the program where the modeling was useful. 

The key tasks an expert would perform in working toward a solution were identified, and a 
modeling session was developed for each of the tools he or she would use while performing that task. 
Altogether there were four modeling sessions, all of which students were expected to watch in the course 
of their work in the program. In each modeling session the expert "thought aloud" about the problem, 
explaining what he was trying to accomplish and how he used the tools within the program to conduct his 
work. The expert modeled numerous strategies, including self-questioning, making connections, 
identifying missing information, forming and recording hypotheses, and taking notes in a list format 
(rather than writing notes in complete paragraphs). 

The Didactic Condition 
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The didactic condition was designed to provide students with all of the same information 
provided in the modeling condition, but without expert modeling. Video segments of the expert popped 
up at the same four points in the program in this treatment condition as in the modeling condition. The 
expert explained tool functionality, then offered learners tips about how to work effectively. The content 
of these tips was determined by examining the modeling sessions to identify the strategies that were 
modeled. Each of these strategies was then listed as a short sentence, and an explanation for it was 
developed. 

As the expert presented each tip, he offered an explanation that tied it to the task at hand, and the 
tip appeared on the screen. For example, when the expert offered the tip "Ask yourself questions," he 
explained that learners may want to stop and ask themselves “Does this tell me something that the species 
needs to survive?” The expert did not actually apply the strategy to the task, and did not model his 
thinking as he is engaged in the problem. 

The purpose of including this condition was to isolate the effects of the modeling from the effects 
of the simple provision of helpful strategies. Because both conditions presented the same strategies and 
explained their usefulness, it is possible to determine if simply cueing students to use certain strategies 
impacts their performance to the same degree as expert modeling of these strategies. A difference in 
outcomes between these two condition groups could not be attributed to a difference in amount of support 
offered; it could only be attributed to the method used in presenting that support. 

The Help Condition 

In the help condition, the expert explained the functionality of the tools provided in the program 
as students accessed them. As in the other conditions, there were four help sessions. These four help 
sessions contained the same information about tool functionality offered in the previous two conditions, 
but the expert provided no advice on how students should work. 

The content of these help sessions was developed by examining what information learners in the 
modeling condition were given about the functionality of each of the tools used during the session. The 
help condition was designed to reflect the typical type of help provided in a software program; such help 
supports learners in the efficient use of a tool without suggesting the reasons they should use the tool or 
the cognitive processes that would make for effective problem-solving. Providing this content to students 
in one condition made it possible to isolate the effects of the introduction of problem-solving strategies 
from the effects of support for understanding tool functionality. 



Dependent Variables 

Three dependent variables were examined for the effects of the expert tool: learners' actions 
wi thin the program during periods of self-directed study, their rationale for the solutions they offered to 
the central problem of the PBL unit, and their attitude toward the program and the expert. 

Self-directed Study 

Extensive periods of self-directed study are an integral element of problem-based learning. The 
expert tool in Alien Rescue is designed to impact the way students work during self-directed study. The 
program features an extensive audit trail that keeps track of the actions learners take and the artifacts they 
construct. This audit trail was examined in depth to determine whether the expert tool had any effect on 
how students used two other tools in the program: their online notebooks and the mission statements for 
the probes they designed. These tools were examined both for how often students accessed them and for 
the contents of the statements students recorded in them. 

To identify the types of differences that could be expected, the scripts for the expert modeling 
segments were examined and a list of modeled behaviors that would leave a trail in learners’ work was 
generated. Differences in the way learners performed these tasks would support the claim that the 
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hypermedia based expert modeling affected learners’ actions during self-directed study. Ten measures 

were used to examine learners’ actions during self-directed study: 

• Number of parsed statements included in the notebook by the seventh day. By this point in the 
program, students had had the opportunity to hear at least the first two expert sessions, each of which 
contained support for the use of the notebook. The contents of students' notes were parsed into 
statements that each contained a single fact or concept Students' score was the total number of 
statements recorded. This was used to determine if the treatment condition impacted the number of 
notes taken. 

• Number of statements in the notebook that were relevant to the solution development by the seventh 
day of class. Relevant facts were defined as those that could help the learner to match the needs of 
alien species to the characteristics of worlds in our solar system. For example, the note "lo has a thin 
atmosphere" was scored as relevant because several species need atmospheres. In contrast, the note " 
Io wobbles a little in its orbit of Jupiter" was scored as irrelevant; while interesting, this fact is not 
related to the needs of any of the species, and could not help learners to determine if any of the 
species might be able to live there. The learner’s score on this measure was the total number of 
relevant statements he or she recorded, and was used to examine the impact of the treatment condition 
on learners’ ability to identify pieces of information useful to the development of a solution. 

• Percentage of statements in notebooks that were relevant to the solution development by the seventh 
day of class. This figure was calculated by dividing the number of relevant statements by the total 
number of statements students recorded. The purpose of this score was to collapse the above two 
measures into a single variable that would reflect the degree to which students were able to 
distinguish pieces of information useful for development of a solution from other facts that were not 
helpful. A high percentage score indicated that a student, no matter how many facts he or she 
recorded, seemed to understand which facts would be useful in solving the problem. A low score, on 
the other hand, indicated that the student tended to record a variety of facts without attention to which 
ones were of use in developing a solution. 

• Use of lists in the notebook by the seventh day of study. This is considered an expert strategy 
because items in a list tend to be more succinct than complete sentences, and because lists facilitate 
later review of information better than the same information recorded in paragraph form. Students' 
use of lists were scored on a zero to three-point scale, and this measure was used to indicate how the 
expert impacted the way they use tools. 

• Number of meaningful sections created in the notebook. This is considered an expert strategy because 
organizing notes into sections helps the learner to structure information in a meaningful way and 
makes those notes more readily accessible for later review. A section was considered meaningful if it 
had a title of a species, a world, or a relevant science topic, plus at least ten words of notes, or if it had 
more than thirty words of notes related to one of these topics. Students' score on this measure 
equaled their total number of sections, and the scores were used to examine the effect of the expert on 
students’ tool use. 

• Number of times notebook was accessed. The expert modeled opening and closing the notebook 
frequently, so as to move flexibly between the notebook and the tool it covers. This is considered an 
expert skill because it indicates flexible and coordinated use of tools. The program audit trail 
provided a count of the total number of times students opened their notebooks. Students’ score on 
this measure was equal to the number of times they opened their notebooks, and was used to examine 
the effect of the expert on students’ tool use. 

• Number of probes designed. The expert modeled recording questions in the probe mission statement 
that reflect the information about a world that is missing from the solar system database but that is 
relevant to the solution development He further modeled selecting instruments for the probe based 
on the questions recorded in the probe mission statement While the expert did not suggest that 
students should limit the number of probes they design, one expected effect of this modeling was that 
students in the modeling condition would be more planful in their design efforts than students in the 
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other conditions. Since this is time-consuming, students in the modeling condition would be expected 
to design fewer probes. The number of probes students designed during the entire course of the 
program was counted and this number served as their score on this measure. 

• Percent of probes designed that had meaningful mission statements. A meaningful mission statement 
is one that identifies specific pieces of data that the probe should collect. Mission statements that 
include questions like “Does Titan have a magnetic field?” or statements like “check for oxygen in 
atmosphere” are considered meaningful. Missions statements that contain vague statements such as 
“see if the Wroft can live there” or where the content is gibberish or missing altogether are not 
considered meaningful. The expert modeled writing meaningful probe mission statements in the third 
modeling segment The number of probes containing meaningful mission statements were divided by 
the total number of probes the learner designed to derive a percent score for this measure. 

• Number of times probe mission statement was accessed. The expert modeled referring back to the 
probe mission statement to guide the probe design process. This is considered an expert strategy in 
that the learner works to the specifications he or she developed in the mission statement. The 
program audit trail recorded the number of times students accessed the mission statement, and the 
total served as a score on this measure. This was used to examine the effect of the expert on students’ 
tool use. 

Problem Solution and Rationale 

A form is provided within the Alien Rescue environment for students to enter their 
recommendations for the problem solution for each of the six species. This form contains six sections, 
one for each species. Each section has two parts: a drop down menu students use to select a world for the 
species, and a text field where they type a rationale for their decision. Use of this form is modeled by the 
hypermedia based expert. In class, students shared the responsibility for developing rationales for each of 
the species, with each individual student responsible for writing the recommendation and rationale for 
two of the species. 

Students’ recommendations for each of these two species were evaluated separately using a 
rubric, then the scores for each were totaled. The criteria included in the rubric are explained below. 

• Correctness of the proposed solution: zero to two points. For each species, there are one or two 
“best” choices, and one or two other “acceptable” choices. Students received two points if they 
recommended a best choice, one point if they recommended an acceptable choice, and zero points for 
any other choices. 

• Inclusion of a topic sentence: zero to one point The expert modeled writing appropriate topic 
sentences in the fourth modeling segment as a method to orient the reader to the position being taken. 
Essays that included a topic sentence scored one point, and essays that lacked them scored zero. 

• Number of supporting details: zero to six points. Students scored one point for each supporting detail 
that they offered, to a maximum of six points. Details in excess of six did not score additional points. 
This was done to prevent the inclusion of supporting details from overwhelming the other aspects of 
the task. This criterion accounts for fifty percent of the total possible points students could score on 
these essays. This was deemed appropriate by the raters because much of the modeling provided by 
the expert was aimed at including appropriate supporting details in the rationale and because the 
effectiveness of a rationale for a decision is largely dependent on the number of specific supporting 
details offered. 

• Inclusion of potential drawbacks: zero to one point If the student acknowledged that a world may 
have one or more drawbacks, one point was scored. Students also scored a point if they state that 
there are no drawbacks or disadvantages for the selected world. The latter scored because it indicated 
that the learner has at least considered the possibility that a drawback exists. Failure to mention 
potential drawbacks scored zero points. 

• Overall persuasiveness of rationale: zero to two points. This was the most subjective part of the 
rubric and gave the raters the opportunity to evaluate the solution and rationale holistically. Through 
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discussion and group scoring of some of the essays, the raters identified some characteristics of 
persuasive responses. The raters agreed that persuasive responses do not contain irrelevant 
statements, make ties between the needs of the species and the selected world, do not use vague 
supporting details, and generally have nothing that detracts from the purpose of the essay. While the 
number of supporting details included affects the overall degree of persuasiveness of an essay, the 
raters made every attempt to focus on other strengths and weaknesses of the essay in order to 
determine the score for this item in the rubric. 

Scores for each of the recommendations ranged from a minimum of zero points to a maximum of 
12 points; by totaling the scores on the two recommendations each student wrote, scores for each student 
ranged from zero to 24 points for this measure. Two raters, one of whom was blind to the students’ 
treatment conditions, scored students’ recommendation essays. An orientation was held in which the 
raters discussed how to score the essays, and the raters scored 1 0% of the responses together. Raters 
scored the remaining responses independently, then exchanged responses and reviewed the scoring. 
Discrepancies were discussed until agreement was reached on all of them, resulting in an interrater 
reliability of 100%. 

Attitude toward the Learning Environment 

Students’ attitude toward the Alien Rescue environment was examined using the Attitude toward 
Learning Environment questionnaire, which was administered in the final week of the study. This 19 item 
questionnaire uses a 5-point Likert scale with responses ranging from 1 for Disagree Strongly to 5 for 
Agree Strongly. Four subscales are included. These subscales are shown in Table 1 with related 
reliability information. 



- Insert Table 1 Here - 

There have been several phases in the construction of The Attitude toward Learning Environment 
questionnaire, each of which has resulted in changes. It is based on a questionnaire originally developed 
by S. Williams to assess attitude toward a mathematics software environment [15]. In the next phase it 
was modified by D. Williams to assess attitude toward the Alien Rescue environment [ 16]. The scale was 
further modified to assess attitude toward the expert modeling within the Alien Rescue environment as 
well as toward the environment as a whole. This version was pilot tested, changes were made based on a 
reliability analysis of each of the subscales, and this revised version was used in this study. 

Procedures 

The study was conducted during students’ regular science class times, which were daily 45- 
minufe periods. Students were engaged in the program for approximately fifteen class periods. 

Each student worked at his or her own computer. Students were encouraged to work together to 
share the information they discovered and to ask each other for help. They were not, however, assigned 
partners or told they had to divide up the work in any way. 

The researcher served as the teacher during the study. In this role, she led whole class 
discussions, encouraged peer interactions, questioned students about their probe designs and provided 
funding for students to launch the probes, and did troubleshooting on technical problems. The regular 
classroom teacher remained in the computer lab with her classes. 

On the first day of the study, learners watched the opening scenario of Alien Rescue then began 
exploring the environment on their own. Over the following two and a half weeks students engaged in 
self-directed study, spontaneous collaboration, and whole class discussions. Whole class discussions 
occurred at the beginning of most classes and tended to last between eight and fifteen minutes. During 
these discussions the learners shared information they had discovered, asked questions of their classmates 
to find needed information, discussed scientific topics, and modeled their problem-solving process for 
their peers. The remainder of the time each day was spent with students working at their computers in a 
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self-directed manner. Many of the students collaborated, sharing information and sometimes dividing up 
responsibility for finding information. Students were asked to listen to the expert and were told that they 
would be asked at the end of the study to share their opinions of the expert. The attitude questionnaire 
was administered in the last week of the study. 



Data Analysis 

To answer the first question, “Does hypermedia based expert modeling affect learners’ work 
during periods of self-directed study in PBL?” a one-way ANOVA was calculated on each of the ten 
measures described in the section on self-directed study, with treatment condition assignment as the 
between groups variable. 

For the second question, “Does hypermedia based expert modeling affect the quality of the 
rationale students develop for their solution to the central problem posed by the PBL unit?” learners’ 
scores on the problem-solving solution and rationale measure were analyzed with a one-way ANOVA, 
with treatment condition as the between groups variable. 

The third research question, “Does hypermedia based cognitive modeling affect learners’ attitude 
toward the learning environment?” was addressed using students’ responses to The Attitude toward 
Environment questionnaire. An analysis of variance was conducted to determine if students differed by 
treatment condition on any of the subscales. 



Results 

Self-Directed Study 

The analyses of variance of the self-directed study measures showed an overall significant 
difference among the treatment condition groups in eight of the ten measures. Post hoc analyses were 
conducted on these eight measures using Tukey’s HSD. Table 2 summarizes the differences among the 
three treatment groups. 

These results suggest that the treatment condition affected students’ work during self-directed 
study in a number of ways. First, the contents of students’ notes differed. Students in the modeling group 
recorded significantly more facts relevant to the development of a solution plan than did students in the 
didactic group (p < .01). This suggests that the modeling was effective in helping students to identify 
pieces of information pertinent to the solution and in convincing them to record this information in then- 
notebooks. While the difference between the modeling and help groups on the number of relevant notes 
recorded was not significant ( p = . 14), students in the help group also recorded more irrelevant notes. As 
a result, students in the modeling condition had a significantly higher percentage of notes relevant to 
solution development than students in the help condition. These results indicate that students who 
received expert modeling were better able to distinguish facts that were helpful from those that were not 

The results also show that the expert modeling impacted the way students used the notebook. 
Students in the modeling condition showed greater organization of their notes than students in the other 
conditions, recording them in list format and separating them into sections by topic. They also used the 
notebook more flexibly, opening and closing it more often than their peers in order to coordinate its use 
with the use of other tools. 

Interestingly, the results also show differences between the didactic and help groups in how they 
used the notebook that were not predicted in the study. Students in the help group were significantly 
more likely to record their notes in list format, and though the difference was not significant, they also 
recorded more notes than the didactic group. 

Evidence of the effect of the modeling can also be seen in students’ work on probe design. As 
predicted, students in the modeling condition designed significantly fewer probes {p < .01), suggesting 
they spent more time developing each probe. Even though they designed far fewer probes, they 
developed a greater number of meaningful mission statements than students in the other treatment 
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conditions, with the difference between the modeling and help groups significant at the .05 level. This 
difference is particularly striking when viewed as percentages. Students in the modeling condition wrote 
useful mission statements for 82.5% of their probes, while students in the didactic and help conditions 
wrote useful mission statements for 20.4% and 13.0% of their probes respectively. There was no 
significant difference in the number of times students accessed their probe mission statements, but this 
figure may be deceptive. Considering that students in the modeling condition designed far fewer probes, 
they still accessed the mission statement more often than their peers, suggesting that they used the probe 
mission statement as the expert recommended: as a guide for their selection of instruments to include on 
the probes. Taken together, these results suggest that students in the modeling condition worked in a far 
more planful manner than students in the other conditions. 

Insert Table 2 Here - 



Problem Solution and Rationale 

Mean scores on students’ solutions to the problem posed in Alien Rescue were 14.1, 7.3 and 7.0 
respectively for the modeling, didactic, and help groups. The results of the analysis of variance revealed a 
significant difference on this measure (F = 20. 13,;? < .01). Post hoc analysis using Tukey’s HSD showed 
that the modeling group performed significantly better on this task than both the didactic group (p < .0 1) 
and the help group (p < .01). There was no significant difference between the didactic and help groups. 



Attitude toward the Learning Environment 

Results of the analysis of variance conducted on the students’ responses to the Attitude toward 
the Learning Environment questionnaire showed differences between the modeling condition and the 
other two groups on two of the subscales (see Table 3). Contrary to concerns raised in the review of 
literature, students in the modeling condition liked the expert better than students in the other conditions. 
Post hoc tests showed that the difference between the modeling and didactic groups was significant (p < 
.05). Students in the modeling condition also found the expert more helpful than students in the other two 
conditions, and this time post hoc analysis showed that the difference between both the modeling and the 
didactic groups and the modeling and help groups was significant at p = .01. Overall these results 
suggest that students in the modeling group had a more positive attitude toward the expert than students in 
the other treatment conditions. 

- Insert Table 3 Here - 
Discussion 

This study shows that the cognitive modeling version of the hypermedia based expert tool offered 
more effective support than either of the other two versions. Its impact on learners can be seen in their 
actions within the environment, in the solutions they developed, and in their attitude toward the expert 
tool. 

Several patterns within the self-directed study data suggest ways in which the expert modeling 
affected how students worked within the PBL unit First, the modeling impacted learners’ use of the tools 
within the hypermedia program, with learners in the modeling condition using tools in a manner that was 
more consistent with the expert’s use of those tools than did students in the other two conditions. They 
were more likely to organize their notes into sections, accessed the notebook far more frequently, and 
consulted their probe mission statements more often while designing probes. Watching the expert use 
these tools led students to use tools within the environment in a similar way. Since tool functionality was 
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demonstrated for all of the treatment groups, these differences can only be attributed to the effect of the 
modeling. 

Second, there is evidence that the modeling affected students’ cognition, helping them to work 
more effectively than students in the other treatment conditions. The contents of learners’ notes more 
closely resembled the types of notes taken by the expert than did those of students in the other conditions. 
Students in the modeling condition tended to take notes that were relevant to the development of a 
solution, suggesting that the modeling helped them to distinguish between relevant and irrelevant 
information. Likewise, the higher percentage of meaningful mission statements for their probes suggests 
that students in the modeling condition acted in a more planful manner than their peers in the other 
conditions. 

This difference in cognition also appeared in the problem-solving data, with students in the 
modeling group again outperforming their peers in either of the other conditions. These results suggest 
that the cognitive modeling offered by the expert tool not only led students to apply effective problem- 
solving strategies to their work, it impacted the quality of their reasoning and their ability to present it in a 
convincing rationale for their solutions. In order to write an effective rationale, students had to have 
developed proficiency in handling a variety of tasks throughout their work in the PBL unit and had to 
organize the information they gleaned in a meaningful way. In this way, the more effective self-directed 
study that resulted from the expert modeling led to better problem-solving. 

Of particular interest was the performance of the learners in the didactic condition. These 
learners received the same advice from the expert as learners in the cognitive modeling condition. Yet 
their actions during periods of self-directed study were far less likely to resemble those of the expert than 
were the actions of their peers in the modeling group. In fact, their actions did not differ significantly 
(with one exception, as discussed below) from those of learners in the help group, even though the help 
group received no advice from the expert on how to work effectively. In terms of problem-solving, again 
students in the didactic condition did no better than students in the help condition. These results suggest 
that simply providing tips and examples is insufficient to impact learners’ actions or understanding; 
students do not apply advice and examples offered by an expert unless they see them modeled on 
activities similar to the ones in which they are engaged. 

Taken together, these results show that the cognitive modeling offered by the expert tool used in 
this study supports students in working more competently during PBL. Comparison of the modeling 
condition with the help condition shows that the addition of expert support during PBL can help students 
to work effectively. Comparison of the modeling condition with the didactic condition suggests that it is 
the modeling of expert strategies, rather than the mere presentation of them, which impacts students’ 
work. 

One anomaly appeared in the self-directed study data that cannot be explained by the impact of 
the expert tool. Students in the help condition exhibited some behaviors that more closely resembled 
those of the expert than students in the didactic condition. In one measure, the use of lists in the 
notebook, the difference between the help and didactic groups, at p = . 08 , neared the level of significance. 
Informal classroom observations suggest that students in the help condition tended to interact more with 
their peers than students in the other two conditions, and that they tended to model some helpful activities 
for each other, particularly in notetaking. This difference may therefore be the result of peer modeling, 
though additional research would be needed to test this hypothesis. 

Contrary to the concern raised in the review of literature, this study found no evidence that 
modeling undermines students’ attitudes toward PBL. The results of the Attitude Toward Learning 
environment questionnaire showed that students in the modeling condition enjoyed Alien Rescue just as 
much as students in the other conditions. It further showed students in the modeling condition liked the 
expert better and thought the expert tool was more helpful than did students in the other two conditions. 
These results suggest that, rather than feeling that the expert undermines their control over their activities 
during self-directed study, students may actually like the tool because they feel that it offers effective 
support 
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Unlike most studies that have examined programs employing modeling, this study was designed 
to isolate the effects of the modeling from other aspects of the instructional approach, such as opportunity 
for practice or access to expert suggestions. As a result, it is possible to attribute the differences between 
groups to the effects of modeling rather than to the approach as a whole. The findings of this study are 
consistent with those of Schunk, who found that cognitive modeling of an expert’s strategies was more 
effective than didactic instruction in promoting accuracy in solving division problems [17]. 

These findings have several implications for problem-based learning and hypermedia. 

Implications for Problem-based Learning 

The findings of this study indicate strong support for the inclusion of expert modeling during 
PBL, at least for the population studied here. The modeling impacted the way students worked during 
self-directed study, bringing their actions more into line with the way an expert approached the problem. 
This in turn led to more well-developed solutions to the problem. 

A concern was raised in the review of literature that modeling may undermine students’ sense of 
autonomy and control over their activities during PBL if students believe that they must model their 
actions on those of the expert. This could in turn erode the positive attitude students normally report for 
PBL. However, the results of this study suggest that this was not the case here. Students in the modeling 
condition expressed similarly positive attitudes toward the PBL environment and a better attitude toward 
the expert than students in the other conditions. This study therefore showed that, through the provision 
of an expert cognitive modeling tool, it is possible to support students’ development of problem-solving 
skills without undermining their attitude toward PBL. 

There is also resounding evidence for the effectiveness of PBL with regular education students of 
middle school age. Regardless of treatment condition, students stayed focused on a complex problem for 
three weeks, conducting research to gather the information they needed, sharing their findings with their 
classmates, and critiquing each other’s solution plans. Students showed evidence of complex problem- 
solving strategies. Such results warrant continued efforts in finding ways to make PBL a viable approach 
for a wide range of learners. 

Implications for Hypermedia 

The results of this study show that hypermedia can provide cognitive apprenticeship-style 
support, making it possible to share with the classroom teacher the task of scaffolding students’ work in 
complex learning environments. The non-linear, interactive format of hypermedia means that support can 
be offered “just-in-time” and under learner control. Providing this type of support would be 
overwhelming for a classroom teacher, making it unlikely that PBL would be used extensively without 
hypermedia support, especially with the types of learners who participated in this study. Through the 
provision of cognitive tools such as the one under investigation in this study, hypermedia can make 
challenging approaches like PBL viable for a wide range of learners. 

In terms of the design of this expert tool, an important distinction must be drawn between two 
conceptions of the role of technology in education. Early in its development, the computer was used as a 
teacher, presenting information, asking questions, and judging responses. Recently there has been a shift 
to a view of technology as a collection of cognitive tools, the purpose of which is to support and extend 
the thinking processes of learners. This enables learners to engage in challenging tasks that would 
normally be overwhelming, and construct a more profound understanding than they would otherwise. 
Jonassen, Peck, and Wilson point out that in that early technology-as-teacher paradigm, both students and 
computers were expected to perform tasks better filled by the other [18]. Students were expected to 
receive, store, and retrieve information, something computers do well, while computers were expected to 
make decisions about learning, a task best left to humans. The result was inert knowledge and passive 
learners. One of the reasons the tool under investigation successfully supported learning within a PBL 
environment may be that it was not designed to make decisions or judgments about students’ actions. It 
left those tasks to the students themselves. Rather, it offered them an expert model of productive actions 
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but left them free to perform as they deemed best Had the expert tool been designed to be “intelligent” 
enough to respond to students’ actions, it may have interfered with learner ownership of the problem- 
solving process. In this light, an essential feature of the expert tool is that it function as a cognitive tool, 
under learner control, extending learners’ thinking processes without controlling how they participate in 
complex learning environments. 

Computer-based cognitive apprenticeship style support is normally seen as the domain of 
intelligent tutoring systems (ITSs), which can diagnose learners’ actions and provide coaching. The 
success of the expert tool examined in this study suggests that cognitive apprenticeship can also inform 
the design of some reasonably “low-tech” tools that may be more appropriate in some situations than 
high-powered ITSs. Rather than investing in costly ITSs, inexpensive alternatives may be effective in 
scaffolding students’ learning, especially for novices where coaching is not the most effective type of 
support or for situations where learner ownership of the process is of paramount importance. 

The effectiveness of this tool in a PBL environment suggests that it may be capable of facilitating 
other challenging instructional approaches, though further examination is necessary to determine the 
appropriate conditions for its use. PBL may differ from other approaches in that it, while offering 
students choices within a complex environment, presents a limited environment with a limited number of 
useful activities. That is, it is possible to determine beforehand the types of activities students will need 
to engage in to solve the problem, and to develop support for those activities. The broader the range of 
possible student actions, the more general the supports would have to become in order to remain relevant. 
The strategies modeled by the expert in this study were highly relevant to a very specific problem; 
because they were immediately useful and understandable within the narrow context of the problem, they 
may have been more meaningful than general strategies modeled within a broad context 



Further Research 

This study focused on the use of PBL with regular education middle school students. The tool 
investigated here was designed to offer enough support so as to make PBL a viable approach for this 
population. Further research is necessary to determine if the cognitive modeling of an expert’s activities 
is beneficial to learners who have traditionally engaged in PBL, namely mature learners and the gifted. 

This study examined the impact of the expert tool on individual performance. However, a central 
focus of the constructivist framework that drives the current interest in PBL is that cognition and expertise 
are distributed across individuals and tools working together within a context, and that learning is an act 
of enculturation into a community’s practices. Further research is needed to begin to understand the ways 
in which tools such as the one examined here impact groups of learners working together to accomplish 
the tasks of a community. The anomaly seen in the results on self-directed study, in which peer modeling 
may have accounted for the greater use of expert strategies by the help group than the didactic group, 
suggests the importance of the interactions of learners within the environment created by this PBL unit. 
One of the most powerful effects of the expert tool may lie in its potential to impact a few students, who 
then impact the performance of the entire group. To gain a deeper understanding of the effects of 
cognitive tools, it is necessary to consider both the individual and the group as appropriate units of 
analysis. 
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Table 1 : Reliability of Subscales in the Attitude toward Environment Questionnaire 



Subscale 


Number 
of Items 
in 

Subscale 


Reliability 
Scores from 
Pilot Test After 
Modifications 


Reliability 
Scores from 
Current Sample 


Helpfulness of Expert 


5 


.89 


.90 


Enjoyment of Expert 


5 


.79 


.86 


Enjoyment of Alien Rescue. 


4 


.83 


.77 


Educational Value of Alien 
Rescue. 


5 


.77 


.70 
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significant at the .01 level 
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Table 3: Means, Standard Deviations (in parentheses) and Analysis of Variance for 
Subscales of the Attitude toward the Learning Environment Questionnaire 



Subscale 


Modeling 

Condition 


Didactic 

Condition 


Help 

Condition 


F 


Enjoyment of Expert 


3.4 


2.7 


3.0 


4.18* 




(-85) 


(-82) 


(-81) 




Helpfulness of Expert 


4.0 


2.6 


3.1 


11.56** 




(-83) 


(1.04) 


(.81) 




Enjoyment of Alien 


3.9 


4.3 


4.2 


2.30 


Rescue 


(.87) 


(.56) 


(.52) 




Educational Value of 


3.9 


4.2 


4.0 


1.40 


Alien Rescue 


(.65) 


(-52) 


(.63) 


- 



Note: N = 66 for all subscales 
♦significant at the .05 level 
♦♦significant at the .01 level 
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